Enhancing Decision Tree based Interpretation of Deep Neural Networks through L1-Orthogonal Regularization

One obstacle that so far prevents the introduction of machine learning models primarily in critical areas is the lack of explainability. In this work, a practicable approach of gaining explainability of deep artificial neural networks (NN) using an interpretable surrogate model based on decision trees is presented. Simply fitting a decision tree to a trained NN usually leads to unsatisfactory results in terms of accuracy and fidelity. Using L1-orthogonal regularization during training, however, preserves the accuracy of the NN, while it can be closely approximated by small decision trees. Tests with different data sets confirm that L1-orthogonal regularization yields models of lower complexity and at the same time higher fidelity compared to other regularizers.Comment: 8 pages, 18th IEEE International Conference on Machine Learning and Applications (ICMLA) 201

Die neue anatomische Flügelplatte für osteoporotische Azetabulumfrakturen: biomechanische Testung

Ziel dieser Arbeit war die biomechanische Testung der neu entwickelten Azetabulum Flügelplatte im Vergleich mit einer herkömmlichen Osteosyntheseplatte, sowie eine Fallzahlplanung für folgende biomechanische Studien. Entwickelt wurde diese Platte unter anderem für ein wachsendes Patientenkollektiv von älteren Patienten mit osteoporotischen Azetabulumfrakturen. Bei diesen Patienten über 65 Jahren besteht häufig noch ein hohes Aktivitätsniveau und damit verbunden ein hoher körperlicher Anspruch. Mit der Azetabulum Flügelplatte soll eine osteosynthetische Rekonstruktion zentral dislozierter multifragmentärer Azetabulumfrakturen über gering invasive Zugangswege bei älteren Patienten ermöglicht werden. Das Ziel der Osteosynthese ist bei dieser Patientengruppe im Idealfall die anatomische Rekonstruktion, gegebenenfalls aber auch das Erreichen einer sicheren Frakturheilung als mögliches späteres Implantatlager für eine sekundäre endoprothetische Versorgung. In dieser Studie wurden die biomechanischen Eigenschaften dieser neu entwickelten Platte mit denen einer Becken-Low-Profile-Platte (LP, Stahl; DePuy Synthes®), welche als Standard bei der osteosynthetischen Versorgung von Azetabulumfrakturen eingesetzt wird, verglichen. Die beiden Platten wurden jeweils an einem Einbeinstandmodell mit vorfrakturierten Beckenmodellen aus Kunststoff getestet und verglichen. Die Belastung der Modelle erfolgte durch eine Materialprüfmaschine und die Messung mit einem optischen Messystem, welches auf dem Modell aufgeklebte Markerpunkte erfasst und deren Bewegung im Raum analysiert. Durch standardisierte Algorithmen bei der Versuchsvorbereitung und Durchführung konnten annähernd gleiche Versuchsbedingungen für die Testung der Platten gewährleistet werden. Gemessen wurde die Stabilität der beiden Platten anhand der Bewegung des Frakturspalts und der Torquierung der Frakturfragmente zueinander. Es konnte nachgewiesen werden, dass die Azetabulum Flügelplatte im Rahmen des Versuchssetups keine geringere Stabilität als die aktuell eingesetzte Becken-LP-Platte aufweist. Die Azetabulum Flügelplatte erzielte vergleichbare und zum Teil bessere Ergebnisse, als die herkömmliche Becken-LP-Platte ohne statistische Signifikanz. Vorteile des Designs der Flügelplatte konnten bei der Stabilisierung der frakturierten quadrilateralen Fläche und der Verspannung des Beckenrings beobachtet werden. Eine Überlegenheit der Platte könnte gegebenenfalls in Folgestudien nachgewiesen werden, insbesondere, wenn ein osteoporotisches Knochenmodell verwendet wird. Anhand der Messwerte wurde eine Fallzahlplanung für Folgestudien durchgeführt. Diese ergab eine benötigte Fallzahl von N=15, um mit einer Power von 90% und einem beidseitigen Signifikanzniveau von 0,05 den relevanten Unterschied von der Nullhypothese zu unterscheiden. In ersten klinischen Erfahrungen konnte die Azetabulum Flügelplatte bei mehrfragmentärer Frakturen der quadrilateralen Fläche mit zentraler Dislokation erfolgreich eingesetzt werden. Die Reposition und Fixation konnte bei allen 8 bisher behandelten Patienten über den Stoppa-Zugang in Kombination mit dem ersten Fenster sicher durchgeführt werden. Im drei-Monats Follow-up konnten keine sekundären Repositionsverluste, Infekte oder Anzeichen einer Früh-Arthrose beobachtet werden

Symbolic Reasoning for Hearthstone

Trading-Card-Games are an interesting problem domain for Game AI, as they feature some challenges, such as highly variable game mechanics, that are not encountered in this intensity in many other genres. We present an expert system forming a player-level AI for the digital Trading-Card-Game Hearthstone. The bot uses a symbolic approach with a semantic structure, acting as an ontology, to represent both static descriptions of the game mechanics and dynamic game-state memories. Methods are introduced to reduce the amount of expert knowledge, such as popular moves or strategies, represented in the ontology, as the bot should derive such decisions in a symbolic way from its knowledge base. We narrow down the problem domain, selecting the relevant aspects for a play-to-win bot approach and comparing an ontology-driven approach to other approaches such as machine learning and case-based reasoning. Upon this basis, we describe how the semantic structure is linked with the game-state and how different aspects, such as memories, are encoded. An example will illustrate how the bot, at runtime, uses rules and queries on the semantic structure combined with a simple utility system to do reasoning and strategic planning. Finally, an evaluation is presented that was conducted by fielding the bot against the stock “Expert” AI that Hearthstone is shipped with, as well as Human opponents of various skill levels in order to assess how well the bot plays. Evaluating how believable the bot reasons is assessed through a Pseudo-Turing test

Diurnal variations of BrONO₂ observed by MIPAS-B at midlatitudes and in the Arctic

The first stratospheric measurements of the diurnal variation in the inorganic bromine (Bry) reservoir species BrONO2 around sunrise and sunset are reported. Arctic flights of the balloon-borne Michelson Interferometer for Passive Atmospheric Sounding (MIPAS-B) were carried out from Kiruna (68° N, Sweden) in January 2010 and March 2011 inside the stratospheric polar vortices where diurnal variations of BrONO2 around sunrise have been observed. High nighttime BrONO2 volume mixing ratios of up to 21 pptv (parts per trillion by volume) were detected in late winter 2011 in the absence of polar stratospheric clouds (PSCs). In contrast, the amount of measured BrONO2 was significantly lower in January 2010 due to low available NO2 amounts (for the build-up of BrONO2), the heterogeneous destruction of BrONO2 on PSC particles, and the gas-phase interaction of BrO (the source to form BrONO2) with ClO. A further balloon flight took place at midlatitudes from Timmins (49° N, Canada) in September 2014. Mean BrONO2 mixing ratios of 22 pptv were observed after sunset in the altitude region between 21 and 29 km. Measurements are compared and discussed with the results of a multi-year simulation performed with the chemistry climate model ECHAM5/MESSy Atmospheric Chemistry (EMAC). The calculated temporal variation in BrONO2 largely reproduces the balloon-borne observations. Using the nighttime simulated ratio between BrONO2 and Bry, the amount of Bry observed by MIPAS-B was estimated to be about 21–25 pptv in the lower stratosphere

Deep feature learning of in-cylinder flow fields to analyze cycle-to-cycle variations in an SI engine

Machine learning (ML) models based on a large data set of in-cylinder flow fields of an IC engine obtained by high-speed particle image velocimetry allow the identification of relevant flow structures underlying cycle-to-cycle variations of engine performance. To this end, deep feature learning is employed to train ML models that predict cycles of high and low in-cylinder maximum pressure. Deep convolutional autoencoders are self-supervised-trained to encode flow field features in low dimensional latent space. Without the limitations ascribable to manual feature engineering, ML models based on these learned features are able to classify high energy cycles already from the flow field during late intake and the compression stroke as early as 290 crank angle degrees before top dead center (-290° CA) with a mean accuracy above chance level. The prediction accuracy from -290° CA to -10° CA is comparable to baseline ML approaches utilizing an extensive set of engineered features. Relevant flow structures in the compression stroke are revealed by feature analysis of ML models and are interpreted using conditional averaged flow quantities. This analysis unveils the importance of the horizontal velocity component of in-cylinder flows in predicting engine performance. Combining deep learning and conventional flow analysis techniques promises to be a powerful tool for ultimately revealing high-level flow features relevant to the prediction of cycle-to-cycle variations and further engine optimization

Pollution trace gases C₂H₆, C₂H₂, HCOOH, and PAN in the North Atlantic UTLS: observations and simulations

Measurements of the pollution trace gases ethane (C2H6), ethyne (C2H2), formic acid (HCOOH), and peroxyacetyl nitrate (PAN) were performed in the North Atlantic upper troposphere and lowermost stratosphere (UTLS) region with the airborne limb imager GLORIA (Gimballed Limb Observer for Radiance Imaging of the Atmosphere) with high spatial resolution down to cloud top. Observations were made during flights with the German research aircraft HALO (High Altitude and LOng Range Research Aircraft) in the frame of the WISE (Wave-driven ISentropic Exchange) campaign, which was carried out in autumn 2017 from Shannon (Ireland) and Oberpfaffenhofen (Germany). Enhanced volume mixing ratios (VMRs) of up to 2.2 ppbv C2H6, 0.2 ppbv C2H2, 0.9 ppbv HCOOH, and 0.4 ppbv PAN were detected during the flight on 13 September 2017 in the upper troposphere and around the tropopause above the British Isles. Elevated quantities of PAN were measured even in the lowermost stratosphere (locally up to 14 km), likely reflecting the fact that this molecule has the longest lifetime of the four species discussed herein. Backward trajectory calculations as well as global three-dimensional Chemical Lagrangian Model of the Stratosphere (CLaMS) simulations with artificial tracers of air mass origin have shown that the main sources of the observed pollutant species are forest fires in North America and anthropogenic pollution in South Asia and Southeast Asia uplifted and moved within the Asian monsoon anticyclone (AMA) circulation system. After release from the AMA, these species or their precursor substances are transported by strong tropospheric winds over large distances, depending on their particular atmospheric lifetime of up to months. Observations are compared to simulations with the atmospheric models EMAC (ECHAM5/MESSy Atmospheric Chemistry) and CAMS (Copernicus Atmosphere Monitoring Service). These models are qualitatively able to reproduce the measured VMR enhancements but underestimate the absolute amount of the increase. Increasing the emissions in EMAC by a factor of 2 reduces the disagreement between simulated and measured results and illustrates the importance of the quality of emission databases used in chemical models

Spectroscopic evidence of large aspherical β-NAT particles involved in denitrification in the December 2011 Arctic stratosphere

We analyze polar stratospheric cloud (PSC) signatures in airborne MIPAS-STR (Michelson Interferometer for Passive Atmospheric Sounding – STRatospheric aircraft) observations in the spectral regions from 725 to 990 and 1150 to 1350 cm−1 under conditions suitable for the existence of nitric acid trihydrate (NAT) above northern Scandinavia on 11 December 2011. The high-resolution infrared limb emission spectra of MIPAS-STR show a characteristic “shoulder-like” signature in the spectral region around 820 cm−1, which is attributed to the ν2 symmetric deformation mode of NO3− in β-NAT. Using radiative transfer calculations involving Mie and T-Matrix methods, the spectral signatures of spherical and aspherical particles are simulated. The simulations are constrained using collocated in situ particle measurements. Simulations assuming highly aspherical spheroids with aspect ratios (AR) of 0.1 or 10.0 and a lognormal particle mode with a mode radius of 4.8 µm reproduce the observed spectra to a high degree. A smaller lognormal mode with a mode radius of 2.0 µm, which is also taken into account, plays only a minor role. Within the scenarios analyzed, the best overall agreement is found for elongated spheroids with AR  =  0.1. Simulations of spherical particles and spheroids with AR  =  0.5 and 2.0 return results very similar to each other and do not allow us to reproduce the signature around 820 cm−1. The observed “shoulder-like” signature is explained by the combination of the absorption/emission and scattering characteristics of large highly aspherical β-NAT particles. The size distribution supported by our results corresponds to ∼ 9 ppbv of gas-phase equivalent HNO3 at the flight altitude of ∼ 18.5 km. The results are compared with the size distributions derived from the in situ observations, a corresponding Chemical Lagrangian Model of the Stratosphere (CLaMS) simulation, and excess gas-phase HNO3 observed in a nitrification layer directly below the observed PSC. The presented results suggest that large highly aspherical β-NAT particles involved in denitrification of the polar stratosphere can be identified by means of passive infrared limb emission measurements

Learning of Generalizable and Interpretable Knowledge in Grid-Based Reinforcement Learning Environments

Understanding the interactions of agents trained with deep reinforcement learning is crucial for deploying agents in games or the real world. In the former, unreasonable actions confuse players. In the latter, that effect is even more significant, as unexpected behavior cause accidents with potentially grave and long-lasting consequences for the involved individuals. In this work, we propose using program synthesis to imitate reinforcement learning policies after seeing a trajectory of the action sequence. Programs have the advantage that they are inherently interpretable and verifiable for correctness. We adapt the state-of-the-art program synthesis system DreamCoder for learning concepts in grid-based environments, specifically, a navigation task and two miniature versions of Atari games, Space Invaders and Asterix. By inspecting the generated libraries, we can make inferences about the concepts the black-box agent has learned and better understand the agent's behavior. We achieve the same by visualizing the agent's decision-making process for the imitated sequences. We evaluate our approach with different types of program synthesizers based on a search-only method, a neural-guided search, and a language model fine-tuned on code